1. Field of the Invention
The present invention relates generally to data collection and storage. More particularly, the invention relates to the collection and storage of data from multiple remote locations, and from diverse hardware and software platforms.
2. Description of Related Art
Computing infrastructures have become increasingly complex and difficult to manage. This is particularly true of large-scale storage area networks (SANs), which are often used with centralized storage. SANs are particularly complex because they include computers, networking infrastructure, and large storage arrays working in conjunction with one another in a diverse operational environment. These networks are typically assembled using components from different hardware vendors, each generally providing a separate and non-integrated software package used for management of the devices. Accordingly, management of these networks is particularly complex and difficult.
A need exists for a system that can actively manage data collection in SANs remotely, and with a high degree of automation. A need particularly exists for a distributed system that enables remote, secure data collection in a scaleable way from the diverse operational environments endemic to SANs.
In accordance with one embodiment of the invention, a system is provided for collecting data from diverse data sources at a remote site. The system includes data collection daemons at the remote site, each for retrieving data from one of the data sources and for translating the data into a common format. A remote query agent collects data in the common format from the data collection daemons, and aggregates the data in a serialized data stream. A central query agent at a central collection site receives the data stream from the remote query agent over an encrypted communications link. A database aggregator initiates data retrieval by the data collection daemons in accordance with a predetermined schedule and deposits data received from the central query agent into a collection database.
In accordance with another embodiment of the invention, a method is provided for remote data collection from diverse data sources. The method includes receiving a request at a remote collection site for collecting data from the data sources in accordance with a predetermined schedule; retrieving the data from the data sources in response to receiving the request; translating retrieved data from each data source into a common format; aggregating the data in the common format into a serialized a data stream; and transmitting the data stream over a communications link to a central collection site for storage in a collection database.
The data collection method and system enable efficient, automated collection of data in the SAN environment. Data can be collected from a variety of data sources in many remote locations having disparate vendor hardware and software. The system is scaleable, secure, and vendor and application neutral.